Methods and systems for detecting stroke symptoms

ABSTRACT

A stroke detection system analyzes images of a person&#39;s face over time to detect asymmetric changes in the position of certain reference points that are consistent with sagging or drooping that may be symptomatic of a stroke or TIA. On detecting possible symptoms of a stroke or TIA, the system may alert caregivers or others, and log the event in a database. Identifying stroke symptoms automatically may enable more rapid intervention, and identifying TIA symptoms may enable diagnostic and preventative care to reduce the risk of a future stroke.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. Provisional PatentApplication No. 62/273,735, filed Dec. 31, 2015, which is herebyincorporated by reference in its entirety.

TECHNICAL FIELD

The present disclosure relates to computerized methods and systems fordetecting stroke symptoms.

BACKGROUND

A stroke occurs when a portion of the brain has insufficient blood flow.There are generally two kinds of strokes, hemorrhagic and ischemic.Hemorrhagic stroke occurs when a blood vessel in the brain ruptures orleaks. This can prevent other areas of the brain from receiving adequateblood flow, as well as creating pressure and other injuries in the areaof the rupture or leak. Ischemic stroke occurs when a blood vessel inthe brain becomes at least partially blocked, preventing a full supplyof blood from passing the blockage to reach other areas of the brain.Stroke may also refer to a transient ischemic attack (TIA), where bloodflow to part of the brain is temporarily interrupted, but is restoredwithout intervention. A TIA may be, but is not always, a precursor to astroke. A person who has experienced a TIA is generally considered athigher risk for stroke than someone who has not experienced a TIA.

Although different people may experience stroke differently, symptomsfor hemorrhagic and ischemic stroke, as well as TIA, are generallysimilar. Symptoms may include slurred, garbled, or nonsense speech andphysical asymmetry. For example, a person experiencing a stroke or TIAmay have difficulty holding an arm up in an extended position for even ashort period of time—the stroke victim may need assistance to lift thearm, or may be unable to hold the arm in a raised position, even if theother arm can easily be raised and held in a raised position. As anotherexample, a person experiencing a stroke or TIA may have a visuallyperceptible droop or sagging in the face, shoulders, or other bodycarriage. The droop or sagging may be pronounced, or the asymmetry maybe most noticeable with movement, such as when the person smiles orspeaks. Typically, weakness and/or drooping are one-sided, occurringpredominantly or entirely on the right- or left-side of the body,depending on what part of the brain is affected by the stroke.

Stroke can be treated, however, currently available treatments requireprompt action. Medication for ischemic stroke is best delivered within4-6 hours of the onset of symptoms. Hemorrhagic stroke may require evenfaster intervention, depending on the severity and location of therupture or leak. A TIA by definition does not require intervention,however, recognizing the occurrence of a TIA is important to allow fordiagnostic and preventative care.

BRIEF SUMMARY

This brief summary is provided as a general overview of the moredetailed disclosure which follows. It is not intended to identify key oressential elements of the disclosure, or to define the claim terms inisolation from the remainder of the disclosure, including the drawings.

This disclosure generally relates to systems and method for detectingstroke symptoms. Generally, and without limitation, the method involvescollecting a series of images of a person's face. The system identifiesreference points on the person's face, for example, points along thecheeks, jowls, and/or brow. The system may superimpose an x-y plane overthe reference points in a digital image. The system then monitors imagesof the face of the person over time. Using the x-y plane (and,optionally, a z-axis), the system compares the positions of thereference points over time. If an asymmetric change in the positions ofthe reference points is detected, the system generates an alert that theperson may be experiencing a stroke.

In some aspects, this disclosure relates to a method for detectingstroke symptoms. The method may comprise receiving from a 3D motionsensor a series of two or more images of the face of a person. Themethod may comprise superimposing an x-y plane over a plurality ofreference points related to anatomical features of the person. Themethod may comprise comparing, over time, positions of the plurality ofreference points relative to the x-y plane. The method may compriseassessing a maintenance of symmetry of the plurality of referencepoints.

The method may comprise assessing symmetry about an x-axis, a y-axis,and a z-axis. The method may comprise identifying a minimum asymmetricchange in the positions of the plurality of reference points. The methodmay comprise evaluating whether the minimum asymmetric change ismaintained for a minimum period of time. The method may comprisealerting a designated recipient of an asymmetric change. The method mayfurther comprise superimposing a z-axis over the plurality of referencepoints. The method may further comprise using the z-axis at least inpart to assess the maintenance of symmetry of the plurality of referencepoints over time. The method may comprise communicating the series ofimages of the face of the person to a central monitoring station. Themethod may comprise displaying series of images for a plurality ofpeople being monitored on a primary display at the central monitoringstation. The method may comprise alerting the central monitoring stationif an asymmetric change in the positions of the plurality of referencepoints relative to the x-y plane is identified. The method may comprisedisplaying images of the person for whom the asymmetric change wasidentified on a central monitoring station alert display upon receivingan alert.

In some aspects, this disclosure relates to a system for detectingstroke symptoms. The system may comprise one or more 3D motion sensors.The one or more 3D motion sensors may be located to provide the one ormore 3D motion sensors with a view of the face of a person to bemonitored. The 3D motion sensors may be configured to collect a seriesof images of the face of the person. The system may comprise acomputerized monitoring system. The computerized monitoring system maybe communicatively coupled to the one or more 3D motion sensors. Thecomputerized monitoring system may be configured to identify a pluralityof reference points on the face of the person to be monitored. Thecomputerized monitoring system may be configured to superimpose an x-y-zaxis system over the plurality of reference points on the face of theperson. The computerized monitoring system may be configured to monitorpositions of the plurality of the reference points of the face of theperson on the x-y-z axis system. The stroke detection system maycomprise a computerized communication system. The computerizedcommunication system may be communicatively coupled to the computerizedmonitoring system. The computerized communication system may beconfigured to send an alert to one or more designated recipients if aminimum asymmetric change in the position of the reference pointsrelative to the x-y-z axis system is identified. The computerizedcommunication system may be configured to send an alert if the minimumasymmetric change is maintained for a minimum period of time.

The stroke detection system may further comprise a central monitoringstation. The central monitoring station may be communicatively coupledto the computerized communication system. The central monitoring stationmay be configured to display at least a portion of the series of imagesof the person. The central monitoring station may comprise a primarydisplay. The central monitoring station may comprise an alert display.The alert display may be a dedicated portion of the primary display. Thealert display may be a separate display or series of displays from theprimary display. If the computerized monitoring system detects anasymmetric change in the positions of the plurality of the referencepoints on the face of the person on the x-y-z axis system, thecomputerized communication system may be configured to send an alert tothe central monitoring station. The central monitoring station may beconfigured to move the display of at least a portion of the series ofimages of the person from the primary display to the alert display uponreceipt of an alert.

In some aspects this disclosure relates to computer-readable storagemedia having embodied thereon computer-executable instructions. Whenexecuted by one or more computer processors the instructions may causethe processors to receive from a 3D motion sensor a series of two ormore images of the face of a person. The instructions may cause theprocessor(s) to superimpose an x-y plane over a plurality of referencepoints related to anatomical features of the person. The instructionsmay cause the processor(s) to compare, over time, the position of theplurality of reference points relative to the x-y plane. Theinstructions may cause the processor(s) to assess the maintenance ofsymmetry of the plurality of reference points over time. Theinstructions may cause the processor(s) to identifying asymmetric changein the positions of the plurality of reference points. The instructionsmay cause the processors to alert a designated recipient of theasymmetric change. The instructions may cause the processor(s) todisplay series of images for a plurality of people being monitored on aprimary display at a central monitoring station. The instructions maycause the processor(s) to alert the central monitoring station if anasymmetric change in the positions of the plurality of reference pointsis identified. The instructions may cause the processor(s) uponreceiving an alert to cause the central monitoring station to duplicatethe display of the series of images associated with the alert on acentral monitoring station alert display.

Additional objects, advantages, and novel features of the disclosurewill be set forth in part in the description which follows, and in partwill become apparent to those skilled in the art upon examination of thefollowing, or may be learned by practice of the disclosure.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING

The description references the attached drawing figures, wherein:

FIG. 1 is an exemplary flowchart for a stroke detection system;

FIG. 2 is an exemplary flowchart for a central monitoring station for astroke detection system;

FIGS. 3A and 3B are exemplary configurations for a face monitoring zone;

FIG. 4 is an exemplary display for a stroke detection system;

FIG. 5 is an exemplary configuration view for a stroke detection system;

FIG. 6 is an exemplary configuration menu for a stroke detection system;

FIG. 7 is an exemplary configuration view for a stroke detection system;

FIG. 8 is an exemplary configuration view for a stroke detection system;

FIG. 9 is an exemplary configuration view for a stroke detection system;

FIG. 10 is an exemplary configuration view for a stroke detectionsystem;

FIG. 11 is an exemplary configuration menu for a stroke detectionsystem;

FIG. 12 is an exemplary configuration menu for a stroke detectionsystem;

FIG. 13 is an exemplary configuration menu for a stroke detectionsystem;

FIG. 14 is an exemplary configuration menu for a stroke detectionsystem;

FIG. 15 is an exemplary configuration menu for a stroke detectionsystem; and

FIG. 16 is an exemplary display for a stroke detection system.

DETAILED DESCRIPTION

Some individuals may be at higher risk of stroke than others. There seemto be hereditary components to stroke, so those with a family history ofstroke may have a higher risk than others with no family history ofstroke. Certain medications or medical procedures may increase the riskof stroke, sometimes for a relatively short period. For example, somesurgical interventions can cause blood clots that can break away fromthe surgical site and be swept through the blood vessels to the brain,where smaller blood vessels may trap the clot, causing an ischemicstroke. That risk may be present for days or weeks after the surgery,and then decrease significantly as the surgical site heals and bloodclots are less likely to form. People who have had a stroke before maybe more likely to have a stroke again. For these and other reasons, somepeople may be at higher risk of stroke than others, and may meritmonitoring for stroke symptoms.

A common stroke symptom is an atypical asymmetry in facial features,often described as a one-sided droop. Most people have some asymmetrybetween the left and right sides of their faces (e.g., about animaginary y-axis run down the center of the person's face), but a personhaving a stroke may lose involuntary as well as voluntary musclecontrol, sometimes resulting in a distinct sagging of one or morefeatures on one side of the face. The sagging may be observed in themouth, for example, where one corner of the mouth might be appreciablylower than the other. The sagging may be most noticeable when the persontries to smile or speak, with one side of the mouth responding, and theother side of the mouth unresponsive and/or sagging. Sagging may also beobserved in the eyebrows, eyelids, cheeks, or jowls.

The subject matter of the present invention is described withspecificity herein to meet statutory requirements. However, thedescription itself is not intended to limit the scope of this patent.Rather, the inventor has contemplated that the claimed subject mattermight also be embodied in other ways, to include different steps orcombinations of steps similar to the ones described in this document, inconjunction with other present or future technologies. Moreover,although the terms “step” and/or “block” may be used herein to connotedifferent elements of methods employed, the terms should not beinterpreted as implying any particular order among or between varioussteps herein disclosed unless and except when the order of individualsteps is explicitly described.

As shown in FIG. 1, a system for detecting stroke symptoms 100 mayinclude one or more 3D motion sensors 110. A 3D motion sensor is anelectronic device that contains one or more cameras capable ofidentifying individual objects, people and motion. The 3D motion sensormay further contain one or more microphones to detect audio. The camerascan utilize technologies including but not limited to color RGB, CMOSsensors, lasers, infrared projectors and RF-modulated light. The 3Dmotion sensor may have one or more integrated microprocessors and/orimage sensors to detect and process information both transmitted fromand received by the various cameras. Exemplary 3D motion sensors includethe Microsoft® Kinect® Camera, the Sony® PlayStation® Camera, and theIntel® RealSense™ Camera, each of which happens to include microphones,although sound capture is not essential to the practice of thedisclosure.

As used herein, “a sensor” and “sensors” are used interchangeably in thesingular and plural unless expressly described as a singular sensor oran array of sensors. A singular sensor may be used, or a sensor maycomprise two or more cameras integrated into a single physical unit.Alternately, two or more physically distinct sensors may be used, or twoor more physically distinct arrays of sensors may be used.

A 3D motion sensor 110 may be co-located with a person 120 to bemonitored. The person 120 to be monitored may be monitored in a varietyof environments, including, without limitation, a hospital, a home, ahospice care facility, a nursing home, an assisted living facility, anoutpatient medical care facility, and the like. The 3D motion sensor 110may be positioned where it is likely to capture images of the face ofthe person 120 to be monitored. For example, a 3D motion sensor 110 maybe oriented to take images of a bed, chair, or other location where theperson 120 to be monitored may spend a significant amount of time. The3D motion sensor 110 may be permanently installed, or may be temporarilyset up in a room as needed. The person 120 to be monitored may be underimmediate medical care, e.g., in a medical facility under thesupervision of a medical professional, or may not be under immediatecare, e.g., in a home or other environment, possibly with a caregiver. Acaregiver may be a medical professional or paraprofessional, such as anorderly, nurse's aide, nurse, or the like. A caregiver may also be afriend, relative, individual, company, or facility that providesassistance with daily living activities and/or medical care forindividuals, such as individuals who are disabled, ill, injured,elderly, or otherwise in need of temporary or long-term assistance. Insome instances, the person to be monitored may be self-sufficient andnot under the immediate care of any other person or service provider.

The 3D motion sensor 110 may communicate data, such as images of theperson 120 being monitored, to a computerized monitoring system 130. Thecomputerized monitoring system 130 is a computer programmed to monitortransmissions of data from the 3D motion sensor 110. The computerizedmonitoring system 130 may be integral to the 3D motion sensor 110 or adistinctly separate apparatus from the 3D motion sensor 110, possibly ina remote location from 3D motion sensor 110 provided that thecomputerized monitoring system 130 can receive data from the 3D motionsensor 110. The computerized monitoring system 130 may be located in themonitored person's room, such as a hospital room, bedroom, or livingroom. The computerized monitoring system 130 may be connected to acentral monitoring station 150. The computerized monitoring system 130and central monitoring station 150 may be remotely located at anyphysical locations so long as a data connection exists (USB, TCP/IP orcomparable) between the computerized monitoring system 130, thecomputerized communication system 140 (if separate from computerizedmonitoring system 130), the central monitoring station 150, and the 3Dmotion sensor(s) 110.

The computerized monitoring system 130 may receive data from 3D motionsensor 110. The data may include images of a monitoring zone 300, asshown in FIG. 3A. At step 180, the computerized monitoring system 130may assess whether a face is detected within monitoring zone 300. If aface is not detected within the monitoring zone 300 at step 180, thecomputerized monitoring system 130 may continue to analyze images forfacial features in monitoring zone 300 as long as 3D motion sensor 110continues to transmit data.

If a face is detected within the monitoring zone 300 at step 180,computerized monitoring system 130 may, at step 185, configure a facemonitoring zone 310. Configuring the face monitoring zone 310 mayinclude digitally superimposing an x-y plane over the face detected inthe image data. The face monitoring zone 310 may include a frame 360around the face detected in the image data. Frame 360 is shown in FIG.3A as rectangular, however, other shapes could be used, including,without limitation, squares, circles, ovals, triangles, and irregularshapes. Frame 360 may help the computerized monitoring system 130 “lockon” to a particular person's face. Locking on to a particular face maybe helpful if, for example, a visitor or caregiver may get very near theperson 120 being monitored, such that the system might otherwiseinadvertently switch from tracking the person 120 being monitored,following instead the face of the visitor or caregiver that entered themonitoring zone 300.

In addition to or in lieu of superimposing a frame 360 around the facedetected in the image data, computerized monitoring system 130 maydigitally superimpose over the face an x-y plane, such as y-axis 340 andx-axis 350. The y-axis may be placed generally along the bridge of thenose, roughly dividing the face into two halves, with one eye on eachside of the y-axis 340. The x-axis 350 may be placed roughly halfwaybetween the top of the head and/or hairline and the bottom of the chin.Reference points may be assigned to distinctive features of the face.For example, in FIG. 3A, there are reference points 320 around the eyes,and reference points 330 around the mouth of the person 120 beingmonitored. It should be understood that the selection of the referencepoints may vary with the individual and/or the configuration of themonitoring system 100. For example, if infrared cameras are used, timeof flight analysis may be used to determine the distance of an object orreference point from the 3D motion sensor 110, and/or may be used toprovide symmetry analysis using a z-axis 810, as described below. Asanother example, if bandages or physiological anomalies would complicatethe tracking of routine reference points, alternative reference pointsmay be assigned. The monitoring zone 300, face monitoring zone 310,reference points 320, 330, frame 360, x-axis 350 and/or y-axis 340 maybe configured automatically by the monitoring system 100, may beconfigured automatically by the monitoring system 100 subject toconfirmation and/or modification by a system user, or may be configuredmanually by a system user.

Returning to FIG. 1, at step 190, the computerized monitoring systemdetermines whether a stroke is detected. Step 190 looks for strokesymptoms, in particular, asymmetric changes in the facial features ofthe person 120 being monitored. By comparing facial images over time,computerized monitoring system 130 can use the facial monitoring zone310 and reference points 320, 330 to determine whether asymmetricchanges in the person's face have occurred. FIG. 3B, for example, may becompared to an earlier acquired image in FIG. 3A of person 120. As shownin FIG. 3B, the reference points 320 and 330 are no longer symmetricabout y-axis 340. The asymmetry represents a change from the image inFIG. 3A, where reference points 320, 330 are more symmetrical abouty-axis 340 than they are in FIG. 3B. Most individuals have some physicalasymmetry, and so testing for a change in symmetry may be more helpfulthan testing for absolute symmetry. Alternately or additionally, thecomputerized monitoring system 130 could evaluate deviation fromabsolute symmetry about y-axis 340 and/or a change in position ofreference points 320, 330 relative to x-axis 350. A timer may beemployed to evaluate whether the asymmetry or change in position of thereference points persists for a minimum amount of time, which could helpdistinguish asymmetric facial gestures, like an eyebrow raised inskepticism, from stroke symptoms.

In some embodiments, a certain degree of change in the symmetry orrelative symmetry about the x-, y-, and/or z-axes, such as a change indistance of at least 10%, or at least 3 mm, may be required beforeissuing an alert. These examples are intended to be non-limiting, andany desired percentage or distance may be used. For example, an alarmlimit may be set to require a change in distance of at least 5%, or atleast 10%, or at least 15%, or at least 20% or more. As another example,an alarm limit may be set to require a change of at least 1-5mm.Additionally or alternatively, other measures of variance could be used,such as standard deviations, and measures may be based on absolute orrelative values. Summed variances, such as the sum of the variancesrelative to the x-, y- and/or z-axes, or any subcombination thereof,could be used as a minimum variance threshold for issuing an alert.

On detecting facial features and/or a change in facial featuresconsistent with a stroke symptom, computerized monitoring system 130 maycommunicate the detected stroke symptom to computerized communicationsystem 140. Computerized communication system 140 may be configured tosend an alert of the stroke symptom to one or more designatedrecipients. Computerized communication system 140 may be an integralpart of computerized monitoring system 130 and/or may be implementedusing separate software, firmware and/or hardware, possibly physicallyremote from computerized communication system 140.

When an alert is triggered, the alert may be sent, at least initially,to the person 120 being monitored, to give the person 120 beingmonitored an opportunity to respond before alerting the centralmonitoring station 150 and/or caregiver(s) 170. For example, an audiblemessage may be played in the room where person 120 is being monitored,possibly asking something like, “Are you ok?” or “Do you need help?”.Shown as step 195 in FIG.1, computerized monitoring system 130 cananalyze subsequent image data from 3D motion sensor 110 for gestures,such as a head nod, consistent with a yes or no answer. If 3D motionsensor 110 is equipped with microphones, computerized monitoring system130 can analyze sound data for recognizable words, such as yes, no,help, or even certain extended sounds, such as “oooooohhhhhhhhh,” whichmight be consistent with moaning or other vocalization associated withpain, discomfort, or disorientation.

Central monitoring station 150 may be alerted if no response is receivedat step 195, or if the response is unintelligible or indicates that theperson 120 being monitored wants or needs assistance. Alternately, oradditionally, central monitoring station 150 may be alerted with or evenbefore person 120, so that central monitoring station 150 can determinewhether the apparent stroke symptom detected is, in fact, problematic.On receiving an alert, the central monitoring station 150, or anattendant there, may view live image, video and/or audio feed from the3D motion sensor 110, and evaluate whether the automated observationsare persistent and/or troubling. If person 120 has been alerted by thecomputerized communication system 140, central monitoring station 150 oran attendant there can use the data from 3D motion sensor 110 toevaluate whether a response from person 120 is reassuring or indicatesthat person 120 requires assistance. Central monitoring station 150and/or computerized monitoring system 130 may analyze the response fromperson 120, however, if the response does not include words or gesturesrecognizable by the computerized system, an attendant at centralmonitoring station 150 may be able to interpret the person's response.If needed, the central monitoring station 150 and/or the attendant couldthen approve alert(s) to appropriate caregiver(s) 170 and/or call foremergency assistance (e.g., send a request for emergency medicalservices to 9-1-1 or a similar service local to the person 120).

One or more caregiver(s) 170 local to person 120 can be alerted with oreven before person 120 and/or central monitoring station 150, so thatthe caregiver(s) 170 can assess what is happening in person. Or,monitored person 120, caregiver(s) 170 and the central monitoringstation 150 could all be alerted at the same time. The priority andtiming of alerts to different individuals or stations can be configuredin accordance with the needs and desires of a particular facility,experience with a particular monitored individual or type of patient, orany other criterion of the system owner or user. This is true forinitial alerts as well as continuing alerts (e.g., if stroke symptomsare detected, and no response from person 120 or a caregiver 170 isreceived or observed) or repeated alerts (two or more distinct eventswhere possible stroke symptoms are observed). The priority and timing ofalerts to different individuals may be different for initial,continuing, and/or repeated alerts.

Data associated with alerts may be logged by computerized monitoringsystem 130 and/or central monitoring station 150 in a database 160. Dataassociated with an alert may include, without limitation, the telemetrydata from 3D motion sensor 110 that triggered the alert; buffered datapreceding the telemetry data that triggered the alert; telemetry datasubsequent to the alert; the number and substantive content of an alert;the individual(s) and/or groups to whom an alert was addressed; theresponse, if any, received or observed following an alert; andcombinations thereof.

As shown in FIG. 2, central monitoring station 150 may receive data frommultiple computerized monitoring systems, 130A, 130B, and 130C. Forsimplicity, the computerized communication system associated with eachcomputerized monitoring system is shown as an integrated component ofthe computerized monitoring system. If desired, separate computerizedcommunication systems and/or a shared computerized communication systemcould be used. Computerized monitoring systems 130A, 130B, and 130Creceive data from 3D motion sensors 110A, 110B, and 110C, which are,respectively, monitoring persons 120A, 120B, and 120C. Data received bythe central monitoring station 150 from computerized monitoring systems130A, 130B, and 130C may routinely be displayed on central monitoringprimary display 200. A single primary display 200 may display data frommore than one computerized monitoring system, shown as view 400 in FIG.4. Alternately, primary display 200 may comprise two or more distinctscreens, each of which may display data from one or more computerizedmonitoring systems. As shown, the display for monitored person 120C hasan open configuration window 410, which is described in greater detailbelow.

When the central monitoring station 150 receives an alert from any ofthe computerized monitoring and communication systems 130A, 130B, 130C,indicating that a monitored person 120A, 120B, or 120C is presenting oneor more stroke symptoms, audio and/or alert information for thatparticular person may be displayed on the central monitoring alertdisplay 210. An alert can be presented in a variety of formats. An alertmay be a visual cue on screen at the central monitoring station 150,such as the specific camera view flashing or being highlighted in acolor to draw attention to that display among others. An alert may be anaudible sound (e.g., a voice or alarm type sound) at the centralmonitoring station 150, an audible sound at the computerized monitoringsystem 130 attached to the 3D motion sensor 110, a text message, anemail, turning on a light or even running a program on a computer.Should the central monitoring station 150 receive alerts from more thanone of the computerized monitoring and communication systems 130A, 130B,130C, indicating that a person 120A, 120B, and/or 120C is presenting astroke symptom, the central monitoring alert display 210 may display thevideo, audio and/or alerting information from all such instances at thesame time. If no alert is received by the central monitoring station150, it may be that nothing is displayed on the central monitoring alertdisplay 210. Preferably, all monitored individual rooms can be displayedand visible on the central monitoring primary display 200 whetheralerting or not. When an alert is generated, attention can be drawn tothe particular camera on central monitoring primary display 200 and/or aduplicative display of the alerting camera can be displayed on a secondseparate computer monitor, e.g., the central monitoring alert display210.

An electronic record of any alerts received, any responses to the alertobserved or received, and/or any actions taken by the central monitoringstation 150 can be stored in a database 160.

As mentioned above, FIG. 4 shows an exemplary view 400 for centralmonitoring primary display 200, including video data for multiplemonitored persons 120A, 120B, and 120C displayed on a single screen.FIG. 5 shows an alternative view 500 for central monitoring primarydisplay 200, including image data for only monitored person 120C. Theview includes a skeletal figure 510, which may be identified by centralmonitoring station 130A, and used to track or “lock on to” the person120C. A skeletal figure 510 is shown in FIG. 5, however, alternate imageanalysis could be used, including, without limitation, blob recognition,facial tracking, facial recognition, or the visual or electronicidentification of a badge or transmitter. No zones are marked in theimage of FIG. 5. FIG. 6 shows an exemplary configuration menu 600, withan option 610 for configuring a face monitoring zone, an option 620 forconfiguring other saved zones, and an option 630 to clear all configuredzones.

FIG. 7 shows view 500 as it might appear on selecting a menu option toconfigure one or more zones. FIG. 7 shows a monitoring zone 300generally about the upper torso, shoulders, and head of person 120Alying in a hospital bed. Monitoring zone 300 may be configured by thecomputerized monitoring system 130. For example, monitoring zone 300 maybe defined as a fixed perimeter or volume around the head of person120C, as determined based on analysis using skeleton figure, blobrecognition, and/or facial tracking. If configured by the computerizedmonitoring system 130, a user may be allowed to modify thesystem-configured monitoring zone 300, or a user may be required orallowed to manually configure the monitoring zone 300. The 3D motionsensor 110 may collect image and/or sound data for a broader portion ofa room than just the monitoring zone 300. The computerized monitoringsystem 130 may analyze only data related to the monitoring zone 300,with or without capturing images and/or sound from a broader portion ofthe room. This may reduce total processing capacity required, as themost processing-intensive algorithms (e.g., facial tracking,identification and tracking of reference points) are run on a limiteddata set. Capturing broader image data may help provide context for analert, e.g., at central monitoring station 150. For example, using imagedata from most or all of the room, central monitoring station 150 or anattendant there may determine that it is unnecessary to send an alert toa caregiver 170 if there is already a caregiver 170 in the room andtending to the person 120 being monitored at the time of an alert. Amonitoring zone 300 may also help stroke detection system 100 “lock on”to a person 120 being monitored, and help avoid situations where acaregiver, visitor, or other person who gets very close to the person120 being monitored might be tracked after moving away from person 120.If the other person moves out of monitoring zone 300, but person 120being monitored does not leave monitoring zone 300, stroke detectionsystem 100 will continue to monitor person 120 in monitoring zone 300.

Using facial recognition algorithms, the computerized monitoring system130 may identify key features of the face of person 120C beingmonitored. Key features may include, without limitation, the orbit ofthe eye socket(s), eyebrow(s), eyebrow ridge(s), the nose, the bridge ofthe nose, the mouth, top of the head, hairline, chin, ears, cheekbones,etc. The features used may vary with the kind of technology (e.g.,visible vs. infrared light) and/or prominent or accessible features onperson 120C. Using the key features, the computerized monitoring system130 may center an x-y plane defined by y-axis 340 and x-axis 350 atroughly the center of the person's face. The placement of the y-axis340, for example, may be conditioned on having one eye and at least aportion of the mouth on each side of y-axis 340. The x-axis 340 may beplaced on the location of the upper cheekbone, orbital bone about theeye socket, the apex of the nose, or the like.

Alternately, the position of frame 360 may be determined, e.g. tocircumscribe the head, and the y-axis 340 may divide the frame 360 inhalf vertically, and the x-axis 350 may divide the frame 360 in halfhorizontally. The absolute placement of the x-y plane is not essential,as long as the x-y plane can be consistently positioned over new imagesof person 120, so that the x-y plane provides a constant frame ofreference. Since the x-y plane will be used for comparing the positionof soft tissue features, if the x-y plane is defined by anatomicalfeatures, those features may preferably be bony, cartilaginous, orotherwise unlikely to move, particularly during a stroke or TIA. Forexample, the ears do not typically sag or droop during a stroke or TIA.As another example, the x-y plane might be situated with theintersection between the y-axis 340 and the x-axis 350 on the tip orapex of the nose of the person 120 being monitored. The computerizedmonitoring system 130 may use facial tracking rather than facialrecognition, facial recognition implying that the software attempts toidentify a particular person (e.g., Jane Doe) based on facial features,as opposed to recognizing a particular facial feature (e.g., an eye)using facial tracking. If desired, facial recognition algorithms couldalso be used, e.g., to confirm that the system has “locked on” to theintended person 120 to be monitored; or to confirm the identity of themonitored person 120 for cross-checking records before recordingmonitoring data to an electronic medical record, billing system, or thelike.

The computerized monitoring system 130 may identify soft-tissuereference points on the face of monitored person 120A. As shown in FIGS.3A and 3B, exemplary soft-tissue reference points may generally outlinethe eyes (reference points 320) and/or the mouth (330). Other exemplarysoft-tissue reference points, which could be used with or in lieu of theeyes and/or mouth, include the jowls, flesh along the cheekbone, theneck, and the portion of the neck immediately under the chin. The eyesand/or mouth may be preferred as they are easily identified by facialtracking algorithms and tend to be readily visible even if the person120 to be monitored is wearing a blanket or high-necked clothing.

If monitoring zone 300 and/or face monitoring zone 310 are configured bya user, the user may operate an input device to select a point on animage or video from the computerized monitoring station 130. The usermay draw a perimeter defining a zone freehand, or may drag the inputdevice (such as an electronic stylus or mouse pointer) from one point toanother to define a diagonal axis for the perimeter of the zone. Otherconfiguration options, including drag-and-drop templates and coordinateidentification, could be used. A 2D monitoring zone 300 and/or facemonitoring zone 310 can be operated as a perimeter, or a third dimensionof depth can be specified. As with the perimeter, the computerizedmonitoring system can define or recommend a depth measurement, or theuser can provide the depth measurement. FIG. 8 shows a pop-up menu 800allowing a user to configure or reconfigure the depth of a monitoringzone. The exemplary pop-up menu 800 solicits a depth parameter specifiedin millimeters (mm), however, any desired unit of measure could be used,including, without limitation, centimeters (cm), meters (m), inches,feet, and yards. Setting a depth for face monitoring zone 310 enablesthe use of a z-axis 810, shown in the inset in FIG. 8, in cooperationwith the x-y plane defined by y-axis 340 and x-axis 350. The use of az-axis 810 may provide for more accurate data for analysis, and may,therefore, allow for the detection of less severe asymmetry than ifusing only an x-y plane. For simplicity, z-axis 810 may not be showneven in configuration views, however, the system can collect and analyzedepth data while evaluating symmetry. Changes in the depth or thicknessof a reference point relative to z-axis 810 may indicate sagging, evenif any corresponding change in position observed relative to the y-axis340 or x-axis 350 is not above a minimum threshold level to identify thechange as a stroke symptom. Any described use of an x-y plane couldalternately use an x-y-z volume or axis system, assuming that depth datacollection is enabled.

On setting a depth parameter, and while still in a configuration view,the depth of the monitoring zone may be visible as a label 900, as shownin FIG. 9. The depth of monitoring zone 300 and the depth of facemonitoring zone 310 may be the same, or they may be configured and/orlabeled separately. The depth label 900 may not be visible duringroutine monitoring and/or alert monitoring, so as not to obscure theperson 120 being monitored and/or other activity in any image data from3D motion sensor 110.

FIG. 10 shows another menu 1000 from configuration view 500. As shown inFIG. 10, a user may be permitted to turn monitoring on or off (e.g., by“unchecking” both video feed and audio feed), or to turn off video feedonly, or to turn off audio feed only, if audio feed is available. It maybe desirable to disable audio feed, for example, at central monitoringstation 150, to prevent overlapping audio feeds from becomingunintelligible noise. If voice or word recognition algorithms are used,those algorithms may run at computerized monitoring system 130 even ifaudio feed is disabled at a monitoring station, such as centralmonitoring station 150. On alert or as desired, the audio feed could beenabled for one or more particular person 120 being monitored, e.g., toprovide context for an alert. It may be desirable to disable audioand/or video feed to provide some privacy to the person 120 beingmonitored. For example, it may be desirable to disable audio and/orvideo feed while the person 120 is being examined by a medicalprofessional, or bathed, or while visitors are present. The need forcomputerized monitoring for stroke detection is somewhat reduced whilethe person 120 is interacting with medical professionals, caregivers, orvisitors. However, if desired, the audio and/or video feed can bemaintained even when there are other others with the person 120 beingmonitored.

Although monitoring zone 300 and face monitoring zone 310 may beconfigured and operational, they may not be shown outside of theconfiguration screens for those zones, as in FIG. 10. That is, the zonesmay be configured and operational, but not superimposed on the images ofperson 120, so as to permit an unobstructed view of person 120, e.g., atcentral monitoring station 150 or while configuring other aspects of thestroke detection system.

If the Device menu 1010 in FIG. 10 is selected, the user may see apop-up menu 1100, as shown in FIG. 11. The use of pop-up, drop down,tabular, or other kinds of menus may be recommended based on, forexample, the number and kinds of options associated with a particularmenu. However, different kinds of menus could be presented based on useror facility preferences. Pop-up menu 1100 includes a number of tabs,from which a tab for Alerts 1110 has been selected in FIG. 11. The spacewithin the Alerts window 1120 is blank, indicating that no alerts havebeen configured. If a user selects Add button 1130 at the bottom of theAlerts tab 1110, a new pop-up menu 1200 may appear, as shown in FIG. 12.As shown in FIG. 12, pop-up menu 1200 further includes drop-down menusto configure an alert by specifying an event 1210, an action 1220, and,if applicable, an N/A field 1230. As with the kinds of menus, theparticular words used to describe an event, action, and/or NA field maybe modified to reflect the environment in which the system is beingused, or the facility or personnel using the system or a particularstation. For example, a system, station, or user interface may beconfigured for use in a hospital using clinical terminology. As anotherexample, a remote central monitoring station 150 may have an attendantwho is not a medical professional, and lay terminology might be used inlieu of or in addition to clinical terminology. Family or othernon-professional and/or non-medical caregivers may have access to thestroke detection system 100 and/or serve as an attendant for a remotemonitoring station, and the menus for those users may similarly usedescriptive, non-clinical terminology in addition to or in lieu ofclinical terminology. Different languages could also be used fordifferent interfaces. As shown in FIG. 12, the stroke detection system100 may include monitoring and/or alert functions unrelated to strokedetection, as well as the “Stroke Detected” option presented. Ifdesired, other options may be removed from the drop-down menu tosimplify user configuration choices for users who do not want or needaccess to the other functions. Changes to the menus, including changesto the range of menu options and the terminology used in the menus, maybe configured when the system is installed or when access is provided toa specific user, and may not require or may not be available for furthermodification by routine system users.

On selection of an event 1210 in FIG. 12, the user may be able to selectan action 1220, as shown in FIG. 13. Several of the options relate toalerts, e.g., to provide different audible signals to the 3D motionsensor 110 and/or computerized monitoring system 130; to add or change acolor border to a display of image data; to highlight or flash a displayof image data; to log an alert, as in database 160; to send e-mail orSMS; or to provide other alerts. As shown in FIG. 13, the user haselected to highlight/flash a display of image data if event 1210 occurs,e.g., if symptoms of a stroke are detected. As shown in FIG. 14, N/Afield 1230 may be blank and/or inactive depending upon the event 1210and action 1220 selected. In the example shown in FIG. 14, the option tohighlight/flash an image display does not require further configuration,and so N/A field 1230 is blank and inactive, in that the user cannotinput options for N/A field 1230. However, if the action 1220 was set tosend an alert, for example, N/A field 1230 might become active and allowa user to designate a recipient and/or recipient group to whom the alertshould be sent. If the user desires to send different kinds of alerts todifferent recipients or groups of recipients, multiple alerts could beconfigured, with each alert specifying a different action 1220 (e.g.,send e-mail vs. send SMS) and/or a different recipient. As anotherexample, the N/A field 1230 could be used to specify where to log theoccurrence of an event, for example, if more than one database 160 isavailable to the stroke detection system 100, or if data for more thanone monitored person is stored in the available database(s).

As shown in FIG. 15, after an alert has been configured, theconfiguration view 500 may revert to alert tab 1110, now showing a briefdescription of configured alert 1500 in alerts window 1120. Ifadditional alerts were configured, alerts window 1120 might display aselectable list of configured alerts, including configured alert 1500.Once configured, alerts may be edited or deleted using buttons 1510 or1520, respectively. Edit button 1510 may re-open the configuration view500 as shown in FIG. 14, with the drop-down menus open to receivealternate selections.

FIG. 16 shows a view of image data from multiple 3D motion sensors 110monitoring persons 120A, 120B, and 120C, as might appear on a centralmonitor primary display 200. The configuration window 410 that was shownin FIG. 4 has been closed, providing an unobstructed view of monitoredperson 120C. Depending upon the configuration for primary display 200,each panel 1610, 1620, and 1630 may display live video, intermittentimages (e.g., “still” shots from a video data feed) and/or audio datafor monitored person 120A, 120B, and 120C, respectively.

The various computerized systems and processors as described herein mayinclude, individually or collectively, and without limitation, aprocessing unit, internal system memory, and a suitable system bus forcoupling various system components, including database 160, with acontrol server. Computerized monitoring system 130 and/or centralmonitoring station 150 may provide control server structure and/orfunction. The system bus may be any of several types of bus structures,including a memory bus or memory controller, a peripheral bus, and alocal bus, using any of a variety of bus architectures. By way ofexample, and not limitation, such architectures include IndustryStandard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus,Enhanced ISA (EISA) bus, Video Electronic Standards Association (VESA)local bus, and Peripheral Component Interconnect (PCI) bus.

The computerized systems typically include therein, or have access to, avariety of computer-readable media, for instance, database 160.Computer-readable media can be any available media that may be accessedby the computerized system, and includes volatile and nonvolatile media,as well as removable and non-removable media. By way of example, and notlimitation, computer-readable media may include computer-storage mediaand communication media. Computer-readable storage media may include,without limitation, volatile and nonvolatile media, as well as removableand non-removable media implemented in any method or technology forstorage of information, such as computer readable instructions, datastructures, program modules, or other data. In this regard,computer-storage media may include, but is not limited to, RAM, ROM,EEPROM, flash memory or other memory technology, CD-ROM, digitalversatile disks (DVDs) or other optical disk storage, magneticcassettes, magnetic tape, magnetic disk storage, or other magneticstorage device, or any other medium which can be used to store thedesired information and which may be accessed by the control server.Computer-readable storage media excludes signals per se.

Communication media typically embodies computer readable instructions,data structures, program modules, or other data in a modulated datasignal, such as a carrier wave or other transport mechanism, and mayinclude any information delivery media. As used herein, the term“modulated data signal” refers to a signal that has one or more of itsattributes set or changed in such a manner as to encode information inthe signal. By way of example, and not limitation, communication mediaincludes wired media such as a wired network or direct-wired connection,and wireless media such as acoustic, RF, infrared, and other wirelessmedia. Combinations of any of the above also may be included within thescope of computer-readable media. The computer-readable storage mediadiscussed above, including database 160, provide storage of computerreadable instructions, data structures, program modules, and other datafor the computerized systems. Computer readable instructions embodied oncomputer-readable storage media may be accessible by stroke detectionsystem 100 and/or component(s) thereof, and, when executed by a computerprocessor and/or server, may cause the system to function and/or performthe methods described herein.

The computerized systems may operate in a computer network using logicalconnections to one or more remote computers. Remote computers may belocated at a variety of locations, for example, but not limited to,hospitals and other inpatient settings, veterinary environments,ambulatory settings, medical billing and financial offices, hospitaladministration settings, home health care environments, payer offices(e.g., insurance companies), home health care agencies, clinicians'offices and the clinician's home or the patient's own home or over theInternet. Clinicians may include, but are not limited to, a treatingphysician or physicians, specialists such as surgeons, radiologists,cardiologists, and oncologists, emergency medical technicians,physicians' assistants, nurse practitioners, nurses, nurses' aides,pharmacists, dieticians, microbiologists, laboratory experts, laboratorytechnologists, genetic counselors, researchers, veterinarians, students,and the like. The remote computers may also be physically located innon-traditional medical care environments so that the entire health carecommunity may be capable of integration on the network. The remotecomputers may be personal computers, servers, routers, network PCs, peerdevices, other common network nodes, or the like, and may include someor all of the elements described above in relation to the controlserver. The devices can be personal digital assistants or other likedevices.

Exemplary computer networks may include, without limitation, local areanetworks (LANs) and/or wide area networks (WANs). Such networkingenvironments are commonplace in offices, enterprise-wide computernetworks, intranets, and the Internet. When utilized in a WAN networkingenvironment, the control server may include a modem or other means forestablishing communications over the WAN, such as the Internet. In anetworked environment, program modules or portions thereof may be storedin the control server, in the database 160, or on any of the remotecomputers. For example, and not by way of limitation, variousapplication programs may reside on the memory associated with any one ormore of the remote computers. It will be appreciated by those ofordinary skill in the art that the network connections shown areexemplary and other means of establishing a communications link betweenthe computers may be utilized.

In operation, a user may enter commands and information into thecomputerized system(s) using input devices, such as a keyboard, apointing device (commonly referred to as a mouse), a trackball, a touchpad, a 3D Gesture recognition camera or motion sensor. Other inputdevices may include, without limitation, microphones, satellite dishes,scanners, or the like. In addition to or in lieu of a monitor, thecomputerized systems may include other peripheral output devices, suchas speakers and a printer.

Many other internal components of the computerized system hardware arenot shown because such components and their interconnection are wellknown. Accordingly, additional details concerning the internalconstruction of the computers that make up the computerized systems arenot further disclosed herein.

Methods and systems of embodiments of the present disclosure may beimplemented in a WINDOWS or LINUX operating system, operating inconjunction with an Internet-based delivery system, however, one ofordinary skill in the art will recognize that the described methods andsystems can be implemented in any operating system suitable forsupporting the disclosed processing and communications. As contemplatedby the language above, the methods and systems of embodiments of thepresent invention may also be implemented on a stand-alone desktop,personal computer, cellular phone, smart phone, tablet computer, PDA, orany other computing device used in a healthcare environment or any of anumber of other locations.

From the foregoing, it will be seen that this disclosure is well adaptedto attain all the ends and objects hereinabove set forth together withother advantages which are obvious and which are inherent to thestructure.

It will be understood that certain features and subcombinations are ofutility and may be employed without reference to other features andsubcombinations. This is contemplated by and is within the scope of theclaims.

Since many possible embodiments may be made of the invention withoutdeparting from the scope thereof, it is to be understood that all matterherein set forth or shown in the accompanying drawings is to beinterpreted as illustrative and not in a limiting sense.

What is claimed is:
 1. A method for detecting stroke symptoms, themethod comprising: receiving from a 3D motion sensor a series of two ormore images of a face of a person; superimposing an x-y plane over aplurality of reference points related to anatomical features of theperson; comparing, over time, positions of the plurality of referencepoints relative to the x-y plane; and assessing a maintenance ofsymmetry of the plurality of reference points.
 2. The method of claim 1,wherein the symmetry is assessed about an x-axis, a y-axis, and az-axis.
 3. The method of claim 2, further comprising identifying aminimum asymmetric change in the positions of the plurality of referencepoints, and evaluating whether the minimum asymmetric change ismaintained for a minimum period of time.
 4. The method of claim 3,further comprising alerting a designated recipient of the asymmetricchange.
 5. The method of claim 4, further comprising superimposing az-axis over the plurality of reference points, and using the z-axis atleast in part to assess the maintenance of symmetry of the plurality ofreference points over time.
 6. The method of claim 1, further comprisingcommunicating the series of images to a central monitoring station. 7.The method of claim 6, wherein series of images for a plurality ofpeople being monitored are displayed on a primary display at the centralmonitoring station.
 8. The method of claim 7, wherein, if an asymmetricchange in the position of the plurality of reference points relative tothe x-y plane is identified, the central monitoring station is alerted.9. The method of claim 8, wherein on receiving an alert, images of theperson for whom the asymmetric change was identified are displayed on acentral monitoring station alert display.
 10. A system for detectingstroke symptoms, the system comprising: one or more 3D motion sensorslocated to provide the one or more 3D motion sensors with a view of aface of a person to be monitored, the 3D motion sensors configured tocollect a series of images of the face of the person; a computerizedmonitoring system communicatively coupled to the one or more 3D motionsensors, the computerized monitoring system configured to identify aplurality of reference points on the face of the person, to superimposean x-y-z axis system over the plurality of reference points of the faceof the person, and to monitor positions of the plurality of thereference points of the face of the person on the x-y-z axis system; anda computerized communication system communicatively coupled to thecomputerized monitoring system, the computerized communication systemconfigured to send an alert to one or more designated recipients if aminimum asymmetric change in the position of the reference pointsrelative to the x-y-z axis system is identified.
 11. The system of claim10, wherein the computerized communication system is configured to sendan alert if the minimum asymmetric change is maintained for a minimumperiod of time.
 12. The system of claim 10, further comprising a centralmonitoring station communicatively coupled to the computerizedcommunication system, the central monitoring station configured todisplay at least a portion of the series of images of the person. 13.The system of claim 12, wherein the central monitoring station comprisesa primary display and an alert display.
 14. The system of claim 13,wherein the alert display is a dedicated portion of the primary display.15. The system of claim 12, wherein if the computerized monitoringsystem detects an asymmetric change in the positions of the plurality ofthe reference points of the face of the person on the x-y-z axis system,the computerized communication system sends an alert to the centralmonitoring station, and the central monitoring station moves a displayof at least a portion of the series of images of the person from theprimary display to the alert display.
 16. Computer-readable storagemedia having embodied thereon instructions which, when executed by oneor more computer processors, cause the processors to: receive from a 3Dmotion sensor a series of two or more images of the face of a person;superimpose an x-y plane over a plurality of reference points related toanatomical features of the person; compare, over time, the position ofthe plurality of reference points relative to the x-y plane; and assessa maintenance of symmetry of the plurality of reference points.
 17. Thecomputer-readable storage media of claim 16, wherein the instructionsfurther cause the one or more computer processors to identify anasymmetric change in the positions of the plurality of reference points.18. The computer-readable storage media of claim 17, wherein theinstructions further cause the one or more computer processors to alerta designated recipient of the asymmetric change.
 19. Thecomputer-readable storage media of claim 18, wherein the instructionsfurther cause the one or more computer processors to display series ofimages for a plurality of people being monitored on a primary display ata central monitoring station.
 20. The computer-readable storage media ofclaim 19, wherein the instructions further cause the one or morecomputer processors to alert the central monitoring station if theasymmetric change in the positions of the plurality of reference pointsis identified and, upon receiving an alert, cause the central monitoringstation to duplicate the display of the series of images associated withthe alert on a central monitoring station alert display.